METHOD, APPARATUS AND PROGRAM FOR IMAGE COMPRESSION, AND 
IMAGE READING APPARATUS 



Patent number: 
Publication date: 
Inventor(s): 
Applicant(s): 
Classification: 
- international: 



JP2006279741 (A) 
2006-10-12 

MOMOZONO YUKINOBU 
SEIKO EPSON CORP 

H04N1/41; G06T1/00; G06T9/00; H04N1/46; H04N1/60; H04N1/41; G06T1/00; 
G06T9/00; H04N1/46; H04N1/60 



- european: 

Application number: JP20050098337 20050330 
Priority number(s): JP20050098337 20050330 



Abstract of JP 2006279741 (A) 
PROBLEM TO BE SOLVED: To provide a method, 
apparatus and program for image compression and 
image reading apparatus with which a digital image 
is compressed with high compressibility so as not to 
damage readability of character.; SOLUTION: The 
image compression method includes: a setting stage 
of setting a processing target region on a digital 
image; a grouping stage of grouping pixels 
constituting the processing target region for each 
pixel with little color difference; a selecting stage of 
selecting a replacement target group based on 
feature amounts of the groups; a location 
information generating stage of indicating a location 
of each of the pixels belonging to the replacement 
target group; a replacing stage of replacing the color 
of each of pixels belonging to the replacement target 
group with the color of surrounding pixels; a digital 
image compressing stage of generating image 
compression data by compressing the digital image 
after the color of each of the pixels is replaced; and 
an output stage of outputting color data representing 
the representative color of the replacement target 
group, the location information and the image 
compression data in an associated manner.; 
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Video Compression wrm Storage Repuctiom, Color Rotation, Combined 

Bmmai and Border Filibkmg 

FIELD OF THE IMVEP1TC5M 

Hie pr&sens hry e&tio^ raises g^B^My cosnpp^&io*^ 
More s^^ciSsaliy, the present kwmmm relates to a good g^&iy ^id ®o c*>de£ te^I^^itatio^ 
feat act$i£*res a ^ hit rate video. 

BACKGROUND OF mm INVENTION 

A number of important applications in image processing require a vers" low cost, fast and 
good quality ^ that ^feieyes a good compression 

ratio, Isi pa^ iiBpI^eMtatioB is desij^Me fef low bit rate video 

applications such as video cassette recorders (V CRs)> cable tel0\Hsio^ y cameras^ set-top boxes 
pid.atfcfir c^imier #vices*. 

One way to achieve a faster and lower cost codec iui|>lcmentaticn is to attempt t<> reduce 
the amount of memory needed by a particular cxHnpressioB aigorflfetn, Reduced iriemory (such 
as BAM) is especially desirable ^ such as 

an an integrated circuit (or ASIC). For e^iipfe it can be ptohife M veiy expeus^e to place large 
amoinits of RAM into a small video camera to allow fbr mom efficient coiiipression of images. 
Typically* spmller ainmmts of RAM are used in order to impfonieiit a : p^tieiilar codtec* bm this 
results m a oodee tftst is : : 1^:«fi^e^ : ;aQit of less quality + 

Although riofahle ad^a&ocs harae been made m ^m ^Md s mS ixi particular with JFEG and 
MPEG coding, there are still drawbacks to these tschm^ ffcom & bette 

codec irnpl^eiitHtioo that achi eves a higher compression ratio using ksss memory. For 
marnple, both JPBG and motiori JPEG coding pet&rm Moefc-fc^Modk c<^pre^dn of a frame 
of an image to pxxsckice compressed. i&depetKleut blocks, For the most part, ^csc blocks are 
treated indcpeudeBlly of o^s a&other, In other wards, IPSO cmMug arid other similar fbsms o f 
still toage cofeg end up compressmg a frame at a time without reler^ice to prs^ioBS w 



1 



WO 99/59329 



s&bssqee&t iam^v These teshm gu^ <io &©t take Ml adv^age of the similmfes between 
frames or between blocks of ^■■^^^.^^■.l^m/P^^m^ & mmgmmmm ratio that is not optimal. 

:OCher types of eodm| s&efe as MPEG m&mg nm kmt^mm or mterfield diSfer^clBg lm 
order to compare &^ a&d tfciis achieve a better eaiBj5T^ssion ratio. However in 

orcterto compare frames, at least one fell Irmjie mtist be stored temporary storage fc orfer to 
£0&spare it to Miber 11ms, to p^u&e fee I, B *• aid P frames 

tseeess&ry tit tb§ § tf pe of emlr^ y ^ rmm ved and slore#^fe caa 

begis. The aB^mnt of teapj data f or iram^ eaa fee pmiiibaive store m ii&M* and makes 
si*efe eodee mtplemeBtatio^ ra h^dware impracMcai due to tbe east and tk^ mm of fee extra 
memory needed. In p^ie&ia^ ttese c^dse tm|>!^me^tMio^ m an Integrated ctemi or similar 
device can be simply to expensive tkm to the amo»t of memory implied. 

Pt^vioas efS^rtS iave sitetitpted m ^cile¥e b^ter ooBtpressloB ratios. For emmpku the 
idea of perfeoning (^^tso^a iti tbe ©CT trasisSirm domain tipoft a wfcofe ^ been 
investigate! before at UC Bericel^ and at tbe IJiiiversi ty of Wasbiitpon for a variety of 
applioatio&s stiellm pictorial databases |zoom a M of 

Tkm* it would be desirable to ba-ve a technique ibr aoMeviiig an improved compression 
ratio for video images ^ needing to be used 

by tbe t^hmque. In pattieular. it wouM be des^ 
of memory needed for an lBipfemeafalioh : :otr^ integrated efeiit 

Boundaries between Mocks present dii^ video Jtp&ges^ A 

brief baclcgroimd ou vite images arid a description of some of tbese di£Kctdl3e$ will now be 
described. FIG. I illustrated a prior art ir&d^ sean lines, 

stripes aad blocks fe^ intaj^p any of a vari ety of sonroes 

sndi as a v^deo e^e^ In aaimagtiig system where 

pmgtm^x mm mm^^^^m^ Wm^ ftmm, In systems wtiere int^iaeed sem Is used, 
eacb image 12 represents a field of ixi^^iEOOii, linage 12 may also repeseM otber bxeafaiowm 
of a still im&ge depe^dlag opo^ tie type of scanning being nsed, Infbnn&tion: in frame 12 is 
re|>£esented by any nnsnber of pmzM 14. Each pixel m mm xm^rmmm digitized mibima*ion and 
is oftm repre^t&ed by 8 bits, although eaeh pixel may be represent by any number of bi ts. 



Each scan line 1 6 Includes any number of pixels 1 4, thereby representing a horizontal line 
of infonnation within fras&e 1 2; Typically, groups of 8 feorkoatai- sean lines are organised imo 
a stripe I S, A hlock of iaiorm#» 20 is pixels wide. 

For example, depending upon the standard being used, a block may be SxS pixels, 8x32 pixels, 
or my Mherm&z®. In $*k fasliim m image is 1^ ami these felmfcs are 

tfees^ ^ posss^^ the 

application. M MTS€ video (a television standard mmg interlaced scan), for ex^|?i§, a Jieid of 
mforniatian appears every 60th of a second, a fame (tetefcg 2 fields) appears every 30th of a 
second and the continuous presentation of frames of information produce a picture. On a 
computer mostltor using progressive scan, a frame of information is refreshed on fee screen 
every 30th <>£ a second -^f^odsiw^^-^s^E^ seen a met* 

PIG. 2 S&islra*^ 

de€on^5res$e4:iaiid. presented ^ viewing, Iniagp S0 eonlafes biocits 52-58 laving bofde^ or 
edges between tbeB^sesves 62-68. image 50 ste^s block feoptidaries 62-68 feaving gbosts or 
shadows psfeeltog artlfkcts). Far a variety of pmm $rt bio^Mwrhlock con^^^03| tcofed^cs, 
the block hmm&m%$ 62-68 become visible because the oo^iati^ 

reoogmsed. Although the Mock bomidaries then^eives may not be- visibly these blocking 
artifacts masn^ the feioek boisridades p^enting an iinacecptabfe image. 

One teetoicps tliat is useiM fm ^ornpressing an ini£tge bioek-by- bloefe i§ to a 2-6 
Biorthogonai &m m mmS&m^ ^mmt lines of pixels or mms of blocks. ^..T^M^i&ss^- filter 
is a ^m^jtU^^^^J^s^: transform. In the Bioithop^a! iit^ srn^ and dlfl^ences of each 
pair of pixels are produced as iu: tfeg ^ (or 

tradMonal- '2-6 Biortbogonai filler* the stream of mm mik^ %~ 
x.> ; - -f x^+ j* tfe£ x valpes j^prs^ntiag a stream of incoming pixels from a soasi iine< Sin^ilarly^ tl^e 
stream of difference values are represented by the formula: d E - x# - x 5 - lite acmal lifted 
stream of difference values ifa&t are output along with the stream of sum values are represented 
by the formula 4 ~ W 8 - The 2-6 Biotihogoiial filter is useful because as can be 

seen by the formula for the lifted values " w'% each resnitant lifted value ** W** depends upon a 
previous and a following sum of pairs of pixels (relative to the difference in question). 
Unfortunately, this overlap between block boundaries makes the compression of blocks 
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dependent upon preceding and succeeding blocks and can tome enormously complex to 
implement For example,, in order to process- the edges of blocks co rrectly usmg the abo ve 
is^^ fern steage for 

eott^s^io^. part of the is&^ hroug ht al«f aitd past of the current 

block mnst also he ieit m stooge for the nsxi block to This complexity not only increases 
the size of the memory required to compress an image, but also complicates the compression 

Prior m techniques have attempted to treat blocks independently but have met with mixed 
results. For example, for a 2-6 Biorthogonal inter the value of w* is calculated tssmg the vary 
first sum (%) and the third mm calcnl&fced However, calculation of the very first 'lifted 
value (w,j) proves more drf&tdt because there is no gs&nism mm with which to calculate the 
value if the Mocks ik& : W- IMhm difficulty occurs at the end of a 

lloek when £he final *$$m is to be calculated, hee&use again, tere is no Mm mzm of 
pixels to be used in i^0^^I^MMf^^^0i ii&M Vajtoe if the hioeks ai^ to foe treated 
independency, (Le., ahloek to be i&^ed ind^pstde^y stoiild m& xxpmt iz&om&mmi torn, 
a prsvio^s or sutec^i^g h3^ek.i) 

Qm solution that tie prior art tjses is to dmply s&bstiitite- aseros. for the coeflieients (tbe 
mm values) m these situations if dala values are not known, l&fortimately* this practice 
in^duees di soo^&oiSes m ise i rnage foe tweeu foioek s and Poekteg artila^s omm as shown in 
FfOJ 2/ artiiae^ ofe^ ^fefy due to zero v&hses %eing w^rteil for some vatos io the 
eMciSMoti of i^ T&eMbre, it 

would foe de^^ 

There is a third difficulty associated with processing a video signal which relates to a 
color carrier. Color rotation of color iiifortnatiou in a video signal typically requires intensive 
computations. Color rotation is often required to transform a color signal from one coordinate 
system |orco^ Common coordinate systems are MIS p&r television 

monitors), YIQ (for NT8C television), and YUV (for component video and S video). For 
example, for an image that is in the 'YUV system (as in many drsmng programs), a eosnpiex 
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mm t^mmm mmMm. Such mat^ 

dm^cm For ^sample, mrm color ?mMkm& mqmm jmm mmgmM^ Qmt al the rest 0 a 
coi^pessioii aigoriihn^ &nd oi&m a separ&te^s^ 
5 cote mm km. If hu% prior ait: color splafe 

MO§L Mm& 20 show FIG, 19 

illustrates traroe portia&s 12a am! 12b tfeat ^^s^ vmgm^^fy Y color m&m^atioH assd O 
color iiifermsittofn of Samo 12, & feis example, frmm 1:2 is r^res^teci YUY color 
coordinates common in component video (Y« or himiiiance nifbmiation, not sfcwi). Pixel 
10 vato&s a(U) 752 ami a{V) 754 repress pi^0is in corrcspo&dmg positions of frames 1 2a and 
1 2b, respectively. 

FXG* 20 iltelratas a pnor art tscimicpe 760 for color rot&Mcm of information in frame 12 
kto a ^i£&ront color coordinate system. Back pair of co^es|^i5iiiig;pi^el values 764 (a two 
vector) &ame portions 12b and I2b are mBtolied W a raiaion maMx R 762 to 

15 produce values 766 m the new coordinate system. New values 766 represent She mmm colors as 
values 764v i?M usmg the diflsrsM eoaKli-nate system. Rotation matrkes R well toown 
values for ^orw^feg ^ soordkutte system to another amlar© 2x2 m&tnces for ooxsverting 
to Yf Q or y W¥, Cmw&mm to Mi3B requires a 3x3 rotate* matrix (a ttoe^dimensio^al 
rotation) TMus, rotat^^ 

20 pixel) of a fame. The sheer number of these limltip&aiiosis SBafce color rolatto slow aad 

expensive. Also, the phmi m^McmM cm &e qy$£^ mmpm$tio®i*. 
Tkeiefoi^, it wchjM he desiraMe to fee afelo to pertonn ^olor rotalio^ oa a signal without 
j-eipirMg tke previous amo^its of processhig power and device mm& needed, 

A fourth difficulty m the prior art exists with respept to comprising composite video arid 
25 S video sipi&ls, Le^ sigrials that eo^feiue ooiors aute h&e&shy . M the early days of tslsvisloii 
it wm discovered that the frequency sp^ctFEm of a y aok and white video signal had a large 
BBmbsr of impoimiated tbp<hjs or Wcs^ . Based upon this chseo^y. It was det^med thai a 
color carter of apps^imateiy 3,6 M^^e^^ signal 
that wcmld **€ll tfesse wpop^latsd region M white 
30 signal > Thus. hlae& and wte^ he asMed to a color c&rrkr to produoe a 
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mmpmiWm&m m00ihM% fer the most pmi^M^tmlmmi4 m& to 
fefeffefei w!|& as^fe^-v ^^^ism^^^^M^'i^^ 82 and a Mask wMle sipial 88 
is shown in FIG, 3, 'l^iealy^ ^ sapsal is modi^&ted ^-i^^^git^^ : tsm 

pkmm 84 asd 86 |&^ng f&adratn^^ -9&? opt of pkas^ wife otl^r. Each 

5 phase carries one color for the color slsr*aL Bach phase is the 

ainplhude of each phase i&dicatittg the amplitmie of its p^iicular color. Oombhikig signals 84, 
86 and 88 produces cornposile si gml 82. Using fcnowm tec hrnqnea, tie ^oiBfeisiatto of the two 
co lor sign als Ifexri each phase of the color earner om he eoihbiasd whh the llacfc a-nd white 
(imtesssity) signal to provide the third color . In addition feeea^se the human eye cannot detect 

10 high irespexicy color, the color carrier is often band limited meaning ihM is fe<p€$iey does not 
change greatly, 

it is &feo common to ssmpte a composite video signal at fbet times the color carrier 
&e<jme0cy, often abo&t a 143 MHz sampimg rate. Signal 82 s^ows sample poirits 
iihistrating a foirttees sampling rate for the color eerier signal S**ch a sampling rate allows 
1 5 both the carrier and its two phases to be detected and rneasitred; tikis, the two phases of the color 

earner can fee separated otil 

Prior art teclmiqnes ha^e fbtind it difficult to direeSlv compress siach a eoiis^ 
signal ;82J : Mo$t prior art teohniqites sepatate t^ 

signal heiore eon^resslo^ ¥hiis y sipials 84 , WS as*d 88 mm st he separated om fk>M 
20 signal 82 before compression &f the eomposite signal cm hepr*. This separation of color is 
mpmmm mm tmm eoosMnntg. Mot o&ly ar^ three diflere&t algorithm :typea!Iy needed, but 
extra Mrdw^ m^av he retired. Compression in hardware is often made more cm^pim and 
costly because of the eon^^ite sigaa t ©neprio^« in 
asiM^g by mi3tg passive components outside of the chip that p erforms the compression The 
25 ihrm different signals are thert fed separately to the compression chip, increasing complexity. 
Alternati vely , separation of the eotor signal can be done on-chip bin this requires extremely 
large m&Mpilers which greatly increase the mm of &m chip > 

Iherafih^ eo&ld tonic eompessiQn of a 

composite video signal directly without thcaeed for prior separation of sipmls or excess 
30 hardware. It; ^ouM fee partietd;^ npon an 
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integrated circuit without the need for oil-chip separation, or for large multipliers on-chip. Such 
a technique would also be desirable for S video and component video. In general, any 
combined video signal that includes black and white and color mf&nw&tmn that needs to be 
sepa^ed dwing copier ession mM Mmm§ t Ip suAi j 



5 The handling of the different types of video in compression is a ftftfc area in the prior art 

that couM also benefit from impro ved techniques. There are three major types of video: 
composite video; S video; and component video. Composite video is single signal that includes 
the felaek/whits signal with a color carrier, Mediated on^ are two 

chrominance signals. S video is a compromise between composite video and component video. 

10 S video has two signals* a Y signal lor Maelc and white ii^mjatioii and a single cfeo tninance 
mgnai The single ste op of a eolor earner wife CI and V color signals 

modulated on to she color eajrier. CoB^OBet^t video contains three separate signals. A Y signal 
for hlaek apd cl^niinance one itsf^ for 

e&raminanee two Mtbrmation , When compression of a video signal is perfe^ed on an 

1 5 integrated circuit in the prior art, the identification of one of fee three types of video sigaais and 
preproeessing of that signal is perfooBed of&ehips 8i$m m$ t^fem<ps® have vet to devise an 
efficient compressor* algorithm on a siBg ie efeip that is ahle to identify and to handle any of the 
lht^}^m'M^i^- $® tM oHp itm& M mm&& thcrefexe he desi^abie for a teehniqae and 
app&iMBS h^^^ itself h^^ arid 

20 compress each these signal^ 



To achieve the f#reping. and in aee^rdanee with %m p^p^# W ^t^S^M^w^^ m 
apparatus and iechniq$se for cor^ressimg video images are d^ the above 

diili^liiies ih the prior art, 

25 A first embodhnent of the present inv ention uses temporary compression of portions of an 

image dBring tte eonipfeie se^nenee ^ amount 

of temporary storage nee&si, parlie^ reduces by a &etor of ten the 

temporary storage i^eded far MterSeli and interMnie tr^fe^fe^ed video con^MsBion, fa 
one specific implementation of tins embodiment, incoming image data is processed and 

30 eoinpr essed hloclc-hy^loek and placed in temporary s^m^ for 



comparison with subsequent blocks before the eventual final compression of the hiformatiorL 
Teniporarv block-by -block compression and the temporal coBipre^ioii of these blocks (between 
frames* for example) not only allows for a reduction m the temporary storage needed, but also 
takes advantage of the relationship between associated blocks of an image in order to produce a 

5 better picture when the information is finally decompressed Taking advantage of temporal 

compression also prociiices a Mgfeer compre^ion ratio, lix particular, this technkpe is especially 
useful fer a codec implemented on an Integrated circuit sncli where less temporary on-chip 
storage is needed and the chip ess* be tnade smaller and faster, Imp! enrcniadou of such a 
powerful codec on a relatively small and inexpensi ve integrated circuit provides efficient and 

iQ high quality video compression in a small device as a camera or other consumer goods, 

ip a nutshell, this first embodiment compresses data b?oek~by-hk>ck before comparing one 
block of a first image with its corresponding block in the next succeeding image using a Haar 
transS^rm Tfie re^tti^^ e&eoded wM erUpul in a uK>r^^e^^ 

Prior art techniques do not utilize the ady^fcage of temj^r^Iy co bloefe and storing 

15 it: while waiting for its cbrrespohding block to hfe input For example, iu JFSC3 and motion 

jFfiti cornpressimi video ith ages are generally processed bfe^ ontptst 
in compressed fo^m. There m tm notion of temporari^ 

comp^ire blocks W a pre vions mm$ fe with corresponding blocks o f a s&eeeeding image. O ther 
compression algb^thtti^^ store bloefes m order to 

20 compare a block of a name to its corresponding block in a later frame, however, storage of 

these Mocks on m integrated circuit (or other device) retires tm extraordinary aniount of 
memory which naak^^ the device um^cessarily large and pmvldes a disincentive to perform 
comparison of e^ Advanta-geQ^sly^ the present m veutio^ slore^ blocks in a 

^^ressed $m less memory 

25 is needed on the device to siore these compressed blocks. Also* tem memory bandwidth is 
needed fer tr^isfertiiig th&m compressed blocks between memory and a processing nnil* 

In a specific embodiment^ a block is transformed, quantized, aud encoded before 
temporal storage m % much compressed form. Later, when a corresponding block from a later 
fen^ is |sfe^;l»q^:bfocto 

30 are decoded back into the transform domain. Advantageously, it is not necessary to perform the 
reverse transform on the smmti blocks. after decoding, them. The two blocks may be compared 
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inthe- iransfbrrn doni&nL Cteee t$m two Mocks haye heen compared, tfee resnk is encoded and 
oMpntm a serial Mt strsatn in a g^a%- ^aH^ess^:,fep^ r : 

TMs embodiment greatly reduces the resources retires! m hardware or software lor 
intoftame or imerfield video eoinpres&ioB, The bwenhon allows for the advantageous 

5 comparison of frames or fteMs but obviates the need to temporarily store a complete frame or 

Mi In particular, t&e benefits a^iieved mcliide: iesa tmpo^ storage required (a^ch as less 
RAM on m ASIC); lower ?nexnory feandwidth. r e^ir^^t^ between temporary storage lime* 
•p?BS:OB: a4e^ke md/or fester tbro^gfep^t); redded computations needed for irjterfeame or 
mterlisM eon^&risnns; aseiid wkh many compression schemes* sneh as JPEG* MPEG. H,263 

1 0 and the like, wavelet eompiressiQB schemes, etc.; may be vm& with, my mmSorm ; md may be 

used with a variety of standards smk as progressive mm and h^efeed. scm. Also, mcodixtg of 
toxks can fee doB« : ^ of teeteques. 

Another important advantage over prior art compression devices is that intensive 
opemtions mch m motion c^mp^s0i<>n in .MPEG are not perfermed . IMifce prior art devices 

1 5 such as the AB ¥60 1 av&slahie Srom Analog Devices, Ine , that rebuke multipliers, the present 
mention, uses sbife atni add for imputations. The result .i$,0 ^t^ technic|u.e and.kss space 
required, Also, prior art MPEG compression devices that perform intensive motion 
compensation are much more complex m4 ex|^T*$iv«g- :(doIlm>>w^e} • thm their corresponding 
decompression devices. By contrast, expression M^.:d^mpros^n : m tk&jytpmxii invention 

20 have skm iar compfedties ; a mmpmsmoxi device aecordmg to ttic present invention is relatively 
less complex: and less e^pe^sive than m MPEG compression device. 

As- mentioned above, one inipottam advantage is that earlier frames (or fields or Mocks) 
used. as predictor can be kept almost eotirely in e0mpre«sf€l :: form i^mm^oM the whole process, 
•grmtly reducing RAM req^iremmts, This is especially adv^ntageoiis for implementation on m 

25 Integrated ciwmt such as an ASI€ whore storage -area, cm he. one-half to two-thirds of the total 
area of the elnp. For sample, fer in*^^ Sold buffer of 

approvingly 2fr Kb^m per fkM is needed, $m manner, frame bufes can be greatly 
reduced or avoided altogether. Images can foe reeonmnieted ironi fee compressed data and the 
#fi&^ioing j>erformed on that daia> As hardware for decoding is relatively inexpensive, fbnr or 

30 five frames worth of data could he decoded at one thne. In one alt^ative emhodhnieii!;, 
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